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ABSTRACT 

Sixteen courses were taught over a 2"year period at 
eight universities using Perseus, a HyperCard-based hypermedia 
information system that contains multimedia information about the 
Greek world, in order to examine how type of assignment influenced 
student perceptions about the Perseus interface and learning 
performance. Data collection methods included interviews, 
observations, automatic transaction logging, questionnaires, and 
document analysis. Initially, evaluation was mostly formative to 
provide feedback to the designers; the focus later shifted to 
summative results that addressed how Perseus affects teaching and 
learning* Assignments were classified as either directed (high 
instructor control), undirected (high learner control) or mixed. 
Those assignments that used both instructional types generally 
yielded better ratings; directed assignments yielded higher ratings 
for confidence, and undirected assignments yielded higher ease of use 
ratings. Additionally, performance ratings were correlated with 
interface effects, but not with frequency of use or previous computer 
experienc*** (Contains 13 references*) (AEF) 



Vc it it -it Vf it it it it it it it it it it it it it it it it it it it it it it it it it it iz it it it it it it it it it it it it it it it it it it it it i: it it it it it it it it it it it it it it iz it 

'* Reproductions supplied by EDRS are the best that can be made 

from the original document* 

it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it it 



Directed and Undirected Tasks in Hypermedia: Is Variety the Spice 

of Learning? 



U.S. DEPARTMCNTCF EDUCATION 
Office of Educational Reseirch and improvftment 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERICJ 

O This document has been reproduced as 
received from the person or organization 
Originating it 

C Minor Changes have been made to improve 
reproduction quality 

• Points of view or opmions Stated in this docu 
ment do not necessarily represent official 
OERl position or policy 



GARY MARCfflONINI and DELIA NEUMAN 
College of Library and Information Services 
University of Maryland 
College Park, MD. 20742 USA 
march@umdd,umd.edu mn9@umaiLumd.edu 

KENNETH MORRELL 
Language Center 
Rhodes College 
Memphis, TN, 38112 USA 
morrell@vax,rhodes.edu 



• PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 

Gary H, iMarks 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER lERlCl ' 



Abstract: Questionnaire data from sixteen courses taught over a two year period at eight 
pi universities where Perseus was used was analyzed to examine how type of assignment 

j;:^ influenced student perceptions about the Perseus interface and learning performance. 

Assignments were classified as either directed (high instructor control), undirected (high learner 
oo control), or mixed and student responses were compared. Assignments that used both 

instnicUonal types yielded generally better ratings, although directed assignments yielded 
higher ratings for confidence that needed information was in the cwpus and undirected 
assignments yielded higher ease of use ratings. AdditionaUy, performance ratings were 
correlated with interface effects but not with frequency of use or previous computer experience. 



One of the thorniest problems in applying hypermedia to learning is related to the problem of user 
control. For more than thirty years (e.g., Mager, 1964) theorists have argued that learners who exercise control 
over their own learning are more motivated, can more easily relate new information to their personal knowledge, 
can adjust pacing to personal abilities, and can benefit from learning that more accurately reflects how people 
learn outside of school. In summarizing the mixed results from many studies of learner control, Ross & 
MoTison (1989) noted that learner control is a multi-dimensional construct. For example, high ability learners 
can manipulate instrucUonal as well as presentational aspects of lessons to their advantage, while lower ability 
learners benefit from presentational but not instructional control. 

Hypermedia technology has captured the imagination of cdiicators for several reasons, including its 
potep^m for. expanding learner access to informaUon resources, blurring teaching and learning roles, providing a 
model for associaUve and non-linear thinking, and offering learner control. In particular, since hypermedia allow 
users to decide which link to follow, learner control is provided as an inherent feature of the technology. Based 
on the assumption that leamer control is positive and that hypermedia provides such control, many 
commentators and designers have proposed strategies to leverage leamer control in design (e.g., Kinzie & Berdel, 
1990;). There is a tension, however, between learning that is fully under the control of the leamer (discovery 
leammg) and learning that proceeds according to carefully sequenced and managed activities (training). Duffy and 
Knuth (1989) pointed out that simply exploring a domain or engaging in "goal free" browsing are not sufficient 
acuviues for formal learning settings and argued for careful consideration of what goals hypermedia assignments 
are to achieve. Most pracUUoners discover some middle ground, sealing either on a guided discovery approach 
or lessons planned to achieve some specific objccUve but with the Hcxibility to take advantage of spontancouc 

'^^"S opportuniucs. Interactive technology in general and hypennedia technology in particular have 

^ exacerbated the tension between assuring ourselves that students acquire specific knowledge and skills that 

--p experts find useful and providing opportuniUcs for students to Icam what to Icam by making their own Icaminc 

^ decisions. * 
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As part of the ongoing evaluation of the Perseus project, questions of leanicr control were examined 
through responses of 350 students in 16 sections of courses at eight different institutions on questionnaires 
specific to their experience with the system. This paper provides evidence for giving both directed and undirected 
assignments rather Aan strictly one or the other. 

The Perseus Hypermedia Corpus 

The Perseus corpus is a HypeiCard-based hypermedia information system that contains multi-media 
information about the ancient Greek world. The system includes Greek and English translations of ancient Greek 
literature together with various commentaries. The first commercial version (Perseus 1.0) includes the works of 
Pindar, Aeschylus, Sophocles, Herodotus, Thucydides, Pausanias, and Homer, as well as selections from 
Plutarch, Hesiod^ and Appolodorus. The user can locate particular wwds in the text and have Perseus provide 
definitions. Pull-down menus at the top of the screen allow links to other texts, graphics, or various indexes. 
These are considered implicit links since they require users to initiate hypertext jumps rather than making 
selections from explicit buttons or anchors that are tied to specific information nodes. In addition to the texts, 
the project has collected approximately 16,000 images of more than 2000 objects and sites and Perseus 1.0 
contains about 7000 of those images. These images represent the collocation of artifacts from a large number 
and variety of museums and archaeological sites. 

A variety of tools make Perseus both usable and useful for analysis. For textual analysis, a 
morphological parser and dictionary with associated inverted indexes allow users to highlight a Greek word, 
identify its lemma and find definitions, and quickly see where that word is used in other works. Searches can 
also be conducted on the definitions themselves to allow users to analyze subtle meanings and styles within and 
across authors and works. Although a few individual tools exist for scholars to conduct such analyses today, 
Perseus is unique in providing both aggregation and extension — to the point of redefining how such research is 
conducted. For archaeological analyses, Perseus offers drawings of sites as they are thought to have existed at 
different periods of time. These drawings can be superimposed to provide a dynamic representation of 
developments over time at the sites. Icons on the drawings allow users to display photographs (digitized still 
images) or short videos (analog videodisc clips) of the sites as they appear today. Users can use the system to 
create their own interpretations of ideas by designing annotated "slide shows" or multi-media collages (sec 
Crane, 1988; Crane & Mylonas, 1988; Mylonas, 1992 for fuller descriptions of the Perseus system). 

Although each of these tools individually represents quantitative changes in what scholars and students 
can do. there is a synergy in their integration. For example, the literature student can jump to maps or site 
drawings to clarify and enrich concepts and ideas encountered while reading text. An encyclopedia, an atlas, and 
bibliographic citations are always available with a few moves of the mouse. Users also have the ability to 
create and store "paths" through the corpus or to enter notes and annotations. By developing paths, instructors 
can provide explicit tours through the corpus to illustrate key ideas. These paths can then be assigned to 
students individually or used in lectures to guide discussion. Students can create their own paths, which are 
submitted to instructors as interj^etations or analyses of assigned problems or themes. A more self-directed use 
of paths is to facilitate the study of students* own learning by reviewing previous iraversals of the corpus in 
much the same way as learners review passages in paper-based texts that they have highlighted with markers. 
Perseus empowers users to aeate new assignments and explorations, new interpretations, and new syntheses; it 
offers qualitative changes in what scholars and students can do. Assessing the effects of these new capabilities 
and the new goals they enable requires new and flexible approaches to evaluation and measurement and was a 
primary challenge of the evaluation effort conducted over the past five years (see Marchionini & Crane, 1994; 
Morrell, Marchionini & Neuman, 1993; Neuman, 1992 for other results of the evaluation effort). 

Method 

The methodology used for evaluating Peiseus was multifaceted (see Marchionini, 1989, for the "CAT 
scan" approach to evaluation). In the first several years the main approach was formative (e.g., Flagg, 1990) to 
provide feedback to the designers. In the later years, more effort was focused on summative results that address 
how Perseus affects teaching and learning. Data collection methods included interviews, observations, automatic 
transaction logging, written questionnaires, and document analysis. This paper focuses on results from 
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questionnaires disiribuled to about 350 students in 16 classes at eight universities over a two year period. These 
sites represent four large research universities (three public and one private), and four small liberal arts colleges. 
Table 1 summarizes the different sites, the types of classes in which Perseus was used, and the type of 
assignmeni(s) given. 

Table 1. Sites, Courses, Questionnaires Returned and Task Type 1991-93 

Task/Type 

follow paths (D) 
follow path, write essay (M) 
conduct lookups, create paths, write essay (U) 
follow paths create paths (M) 

review/enrichment (U) 
follow path, write essay (M) 
find info for essay (U) 
follow path, create path (M) 
translations, oral reports (U) 
create paths (IJ) 
follow paths (D) 
follow paths (D) 
follow paths (D) 
follow paths (D) 
follow paths (D) 
follow paths (D) 

The two-page questidnnaire contained both open-ended and closed questions. The 
closed questions took the form of 5-point Likert scales and were grouped into three clusters: 
demographic, interface, and performance. All students in the 16 courses were asked to 
complete a questionnaire at a point in the semester after the Perseus-specific assignment(s) 
were due. The analysis of interest here was based on comparing questionnaire responses by 
type of assignment. The assignments instructors had devised were classified into one of three 
types: directed (D), undirected (U), and mixed (M). Directed assignments asked students to 
follow paths that were defined by the instructor to find specific information and complete a 
worksheet or answer short questions. Note that all directed assignments were given at one 
institution in sections of a course taught by two instructors who each developed distinct 
assignments. Undirected assignments varied but included using Perseus to assist in translating 
passages of Greek texts, conducting word analyses in texts to gather evidence for an 
interpretive essay about ancient Greek values, and creating paths that illustrated themes or 
concep^. When Doth types of assignments were given, the assignments were classified as 
mixed. In the mixed cases, directed path following activities preceded the undirected 
exploratory activities. 
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Results 



In both years, no statistically reliable differences in students' ratings were found across 
students in different institutions (large research, large public, small private). Surprisingly 
there were no statistically reUable differences due to amounts of previous computer experience 
or frequency of using Perseus. There were consistenUy high correlations between students' 
ratings of the interface (e.g., ease of learning, ease of use, number of times disoriented) and 
performance ratings (e.g., confidence in finding information, contribution to learning, value of 
system, satisfaction); but not between demographic ratings (e.g., age, previous computer 
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experience, frequency of Perseus use) and otlier clusters. Thus, one result of the studies is that 
the interface is more closely related to students* performance ratings than is previous 
experience or other demographic variables. 

For the questionnaires collected over tiie 1991-1993 period, one-way analyses of 
variance across assignment type were performed. Table 2 presents summary data for the 
ANOVAs. For each of the questionnaire items, results of the ANOVA are shown either as 
NSRD (no statistically reliable differences) or in the ordered pair(s) of statistically reliable 
differences at the .05 level based on Tukey HSD post hoc analyses. When multiple pairs of 
contrasts are listed, they arc given in. order of smallest to greatest mean effect. For example, in 
1992-93 students who did mixed assignments were statistically reliably more satisfied with 
their learning than those who did undirected assignments, and these differences were smaller 
than the statistically reliable differences between those who did mixed and directed 
assignments. The third column gives results for the combined data from both years. Since the 
1991-92 questionnaire did not elicit some of tiie items on the 1992-93 questionnaire, those cells 
are blank. All effects are reported from most positive to least positive (e.g., for the frequency 
of getting lost, M>D means that die results were statistically reliably more positive for students 
in the mixed groups, i.e., they reported being lost less often than those in the directed groups). 

Overall, the analyses illustrate that task type was associated with generally strong 
effects. In general, the mixed task type yielded more positive outcomes and ratings, with the 
undirected task type showing results that were almost as good in man> cases. These results 
may reflect that students tended to spend more time using Perseus in undirected settings-i.e., 
when they direct their own learning. Furthermore, these results may be an artifact of the 
setting since students who did mixed assignments tended to spend more time using Perseus 
tiian tiiose in otiier groups. This possibility is mitigated somewhat by the low of correlation 
between usage and performance ratings. The results could also be an artifact of the different 
collegiate settings or may suggest that students generally enjoy more positive experiences when 
they dirfct their own learning. 



Table 2. ANOVA orderings across task types by questionnaire responses for 1992-3 and 1991-92 



Question 

Use/Demographic Cluster 

Pfireq (freq of Perseus use) 
Hours (using Perseus) 
Interface Cluster 
Learn (ease of learning P) 
Use (ease of using P) 
Docuse (usefulness of docum.) 
Images (quality of images) 
Lost (frequency) 
Recover (ease of recovery) 
Performance Cluster 
Infonecd (needed info was in P) 
Confid (confidence to find info) 
Assndif (difference of assign) 
Pcontrib (Pcontrib. to learning) 
Infoamt (amt. of useful info) 
Value Geaming value of time) 
Satis (satis, with learning) 
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Many of the results confirm what might be expected, but other results are more intriguing. 
Although these results must be interpreted with caution, the following interpretations seem 
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plausible. It seems reasonable that students who worked on mixed tasks or those who worked in 
exploratory ways spent more time using Perseus than tho,se working on directed tasks. However, 
it was not the case that students in the directed groups used fewer features than those in the 
undirected groups (ANOVA for the number of Perseus features used across the three types of 
tasks yielded an F ratio of 17.04, p=.tX), with students in the mixed groups using statistically 
reliably more features than either of the other two groups and students in the directed group using 
statistically reliably more features than the students in the undirected groups. Students in the 
mixed groups used an average of 8.3 features, while those in the directed and undirected groups 
used averages of 5.7 and 5.0, respectively. Intuition and experience argue that directed tasks are 
easier to learn and to execute than open-ended tasks. However, Perseus is a complex system and a 
single diversion from a path can lead users into unknown tenitory. Students who explore the 
system may actually expect to become lost, since they are not performing highly goal-directed 
tasks, and thus may rate ease of learning and use less critically than those who viewed the 
assignment in a more linear fashion. The differences in assessing ease of learrang and use may 
reflect the fact that students in the directed groups were all from one site, where access to 
workstations was limited and laboratory assistance was minimal. As is reasonable, the results for 
frequency of getting lost and ease of recovery parallel those for learning and ease of use. Results 
for quality of images are mixed: all students rated the images quite highly, and there is little 
variability across or within groups. 

Results for the performance questions illustrate how expectations and task experience 
combme during learning. Students in the directed groups rated the likelihood that the information 
they needed was in Perseus reliably higher than the other two groups, since they followed defined 
paths and accomplished tasks that required them to locate specific information that they knew the 
designers of the assignments had verified would be there. Students completing more open-ended 
assignments, however, had to define their own information needs and then locate relevant 
information by using Perseus finding aids. These students not only worked without knowing in 
advance whether the appropriate information would be there, they were also subject to the many 
ways information seeking can go awry during the definition-of-needs step as well as when using 
such Perseus tools as indexes and word searches. While it is somewhat surprising that directed 
students did not rate the amount of usefiU information more highly, this rating may reflect 
students' lack of awareness of how to locate information beyond that to which thev had been 
directed. ^ 

Ratings for how different Perseus assignments were from usual assignments may suggest 
that students who worked on undirected tasks viewed them as similar to paper-based tasks in 
which they explored a variety of books or articles for information on topics. Directed students 
may have viewed the Perseus assignment as more typically linear and pedantic. 

Finally, and most importantly, those students who used Perseus in a variety of ways rated 
the value of the time they spent with Perseus more highly than students in either of the other two 
task groups. Likewise, they were more satisfied with their learning than those in the other groups 
Discovenng the proper mix of tasks is clearly one of the challenges of teaching, and these results 
seem to reinforce a "variety is the spice of learning" view of instruction. 

Discussion 

As with most issues of pedagogy, the problem of how much learner control is best for 
students using hypermedia requires relativistic rather than absolute solutions. Although people are 
adaptable and will learn under most conditions, high levels of learner control yield the best results 
when learners have good prerequisite knowledge upon which to build or have large amounts of 
time and patience to invest in learning. Flexible learning environments such as Perseus may be 
designed pnmanly to support self-directed learning but also must provide mechanisms such as the 
path tool that allow students to follow carefully designed instnictional paths. 

Rather than simple variet>- of activities, the fact that mixed assignments began with directed 
and then allowed self-directed exploration suggest that a staged progression of increasing learner 
control is supenor to either the "cookbook" or "sink or swim" methods. This conclusion is due 
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both to the psychology of learning and to the nature of the electronic nnediunL The former predicts 
that learner control is niorc engaging and thus more motivating fo- experienced learners. 
Moreover, new type of challenges in the form of assignments serves as stimulation for continued 
engagement-a self-reinforcing process. The latter predicts that learning within a complex system 
is distinct from using it to accomplish tasks and that progressive disclosure or minimalist 
instruction may be the best way to apply such systems (e.g., Carroll. 1990), Since systems such 
as Perseus are highly complex to learn to use and to actually use after system learning has taken 
place (there are a multitude of features and altemative tools for accomplishing any task), an 
incremental experience can minimalize overload and frustration. Taken together, these two notions 
imply that if hypermedia systems are to be effective for learning, they should be applied for 
sustained periods with varied assignments that are staged from directed to undirected progression. 
Variety is the spice of learning. However, the tension between learner directed and teacher 
directed learning is best managed by insuring that the variety is not random. 
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